\documentclass[11pt,english]{article}
\usepackage{babel}
\usepackage{shortvrb}
\usepackage[latin1]{inputenc}
\usepackage{tabularx}
\usepackage{longtable}
\setlength{\extrarowheight}{2pt}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{color}
\usepackage{multirow}
\usepackage[colorlinks=true,linkcolor=blue,urlcolor=blue]{hyperref}
\usepackage[a4paper,margin=2cm]{geometry}
%% generator Docutils: http://docutils.sourceforge.net/
\newlength{\admonitionwidth}
\setlength{\admonitionwidth}{0.9\textwidth}
\newlength{\docinfowidth}
\setlength{\docinfowidth}{0.9\textwidth}
\newcommand{\optionlistlabel}[1]{\bf #1 \hfill}
\newenvironment{optionlist}[1]
{\begin{list}{}
  {\setlength{\labelwidth}{#1}
   \setlength{\rightmargin}{1cm}
   \setlength{\leftmargin}{\rightmargin}
   \addtolength{\leftmargin}{\labelwidth}
   \addtolength{\leftmargin}{\labelsep}
   \renewcommand{\makelabel}{\optionlistlabel}}
}{\end{list}}
% begin: floats for footnotes tweaking.
\setlength{\floatsep}{0.5em}
\setlength{\textfloatsep}{\fill}
\addtolength{\textfloatsep}{3em}
\renewcommand{\textfraction}{0.5}
\renewcommand{\topfraction}{0.5}
\renewcommand{\bottomfraction}{0.5}
\setcounter{totalnumber}{50}
\setcounter{topnumber}{50}
\setcounter{bottomnumber}{50}
% end floats for footnotes
% some commands, that could be overwritten in the style file.
\newcommand{\rubric}[1]{\subsection*{~\hfill {\it #1} \hfill ~}}
% end of "some commands"
\input{/mnt/exp/MyDocs/pypers/style.tex}
\title{Implementing PEP 318 (decorators)}
\author{}
\date{}
\hypersetup{
pdftitle={Implementing PEP 318 (decorators)},
pdfauthor={Michele Simionato}
}
\raggedbottom
\begin{document}
\maketitle

%___________________________________________________________________________
\begin{center}
\begin{tabularx}{\docinfowidth}{lX}
\textbf{Module}: &
	decorators \\
\textbf{Version}: &
	0.5 \\
\textbf{Author}: &
	Michele Simionato \\
\textbf{e-mail}: &
	MicheleSimionato@libero.it \\
\textbf{Licence}: &
	Python-like \\
\textbf{Date}: &
	September 2003 \\
\textbf{Disclaimer}: &
	This is experimental code. Use it at your own risk! \\
\end{tabularx}
\end{center}




\hypertarget{contents}{}\subsection*{~\hfill Contents\hfill ~}
\pdfbookmark[0]{Contents}{contents}
\begin{list}{}{}
\item \href{#using-decorators}{Using decorators}
\begin{list}{}{}
\item \href{#basics}{Basics}

\item \href{#decorating-methods}{Decorating methods}

\item \href{#decorating-classes}{Decorating classes}

\item \href{#adding-magic}{Adding magic}

\item \href{#defining-method-decorators}{Defining method decorators}

\item \href{#defining-class-decorators}{Defining class decorators}

\item \href{#composing-decorators}{Composing decorators}

\item \href{#diving-into-magic}{Diving into magic}

\item \href{#advanced-usage}{Advanced usage}

\end{list}

\item \href{#the-implementation}{The implementation}
\begin{list}{}{}
\item \href{#module-decorators}{Module \texttt{decorators}}
\begin{list}{}{}
\item \href{#id3}{Metaclasses}

\item \href{#classes}{Classes}

\item \href{#functions}{Functions}

\end{list}

\end{list}

\end{list}



%___________________________________________________________________________

\hypertarget{using-decorators}{}
\section*{Using decorators}
\pdfbookmark[0]{Using decorators}{using-decorators}

Having plenty of free time in these days, I have finished an old 
project of mine, the implementation of \href{http://www.python.org/pep}{PEP 318} in pure Python
(2.3).

Here is the rationale:
\begin{itemize}
\item 
some kind of decorator syntax is scheduled to go in Python 2.4,
therefore it is interesting to play with the concept;

\item 
it is nice to play with decorators now, without having to
wait for one year or so;

\item 
it is much easier to experiment with a pure Python implementation
than with a C implementation;

\item 
the implementation can be seen as an exercise on modern Python
programming and may be valuable to people wanting to study the most
advanced new constructs in Python (\href{http://users.rcn.com/python/download/Descriptor.htm}{descriptors}, \href{http://www-106.ibm.com/developerworks/library/l-pymeta2.html}{metaclasses}, 
\href{http://www.python.org/2.3/descrintro.html}{cooperative methods}, etc.)

\end{itemize}


%___________________________________________________________________________

\hypertarget{basics}{}
\subsection*{Basics}
\pdfbookmark[1]{Basics}{basics}

PEP 318 has the goal of providing a nice syntactic sugar for expressions like
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{def~identity(x):}\\
\mbox{~~~~return~x}\\
\mbox{identity=staticmethod(identity)}
\end{flushleft}\end{ttfamily}
\end{quote}

or
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{def~name(cls):}\\
\mbox{~~~return~cls.{\_}{\_}name{\_}{\_}}\\
\mbox{name=classmethod(name)}
\end{flushleft}\end{ttfamily}
\end{quote}

which are pretty verbose. It is clear that having new syntax (as 
for instance the proposed square bracket notation)
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{def~identity(x)[staticmethod]:}\\
\mbox{~~~~return~x}\\
\mbox{}\\
\mbox{def~name(cls)[classmethod]:}\\
\mbox{~~~~return~cls.{\_}{\_}name{\_}{\_}}
\end{flushleft}\end{ttfamily}
\end{quote}

involves changing the grammar and modifying the interpreter at the
C level. This means a lot of work. Fortunately, it is possible to 
have the same effect without changing the Python grammar. 
The idea is to use magic docstrings like this:
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{def~identity(x):}\\
\mbox{~~~~"[staticmethod]"}\\
\mbox{~~~~return~x}\\
\mbox{}\\
\mbox{def~name(cls):}\\
\mbox{~~~~"[classmethod]"}\\
\mbox{~~~~return~cls.{\_}{\_}name{\_}{\_}}
\end{flushleft}\end{ttfamily}
\end{quote}

The implementation is able to recognize such magic docstrings
and automatically converts methods with magic docstrings in 
method decorators.

Decorators are nothing else than a sophisticated kind of wrappers.
The \texttt{decorators} module provides support both for method decorators
and class decorators:
\begin{itemize}
\item 
\emph{Method decorators} are classes taking a single function as input and 
producing a descriptor object as output. \texttt{staticmethod} and 
\texttt{classmethod} are two examples of already existing
method decorators (actually my implementation rewrites them).
A knowledge of descriptors [\hyperlink{id2}{1}] is not needed in order to use the \texttt{decorator} 
module; however it is welcomed for advanced users wanting to implement
custom method decorators.

\item 
\emph{Class decorators} are metaclasses taking a class as imput and returning
a decorated class as output. A good understanding of metaclasses is needed 
in order to be able to write custom class decorators, but no knowledge
at all is required in order to use the pre-defined class decorators
provided by the module.

\end{itemize}

Notice that properties are not decorators according to my definitions, 
since they take four functions as input, \texttt{get, set, del{\_}} and \texttt{doc}.
Whereas the decorators concept could be generalized to the case of
multiple inputs, I don't see the need for such complication, so
properties are not implemented as method decorators. Moreover, I
think it is much better to use them trough a class decorator.

Finally, the module is meant to be extensible; so one could 
define new kind of decorators. For instance, the original version of
the module also had the concept of module decorators; however I have cut 
down that part in order to keep the documentation short.

Admittedly, the implementation 
is not for the faint of heart, nevertheless I have tried to make the 
basic usage easy and simple to understand.
\begin{figure}[b]\hypertarget{id2}[1]
Descriptors are objects with a \texttt{{\_}{\_}get{\_}{\_}} method; they are quite 
sophisticated, but fortunately they have been wonderfully explained by 
Raymond Hettinger already, so  I am allowed to skip on this point ;).
\end{figure}


%___________________________________________________________________________

\hypertarget{decorating-methods}{}
\subsection*{Decorating methods}
\pdfbookmark[1]{Decorating methods}{decorating-methods}

Before talking about the implementation details, I will show
how the \texttt{decorators} module works in practice. The simplest and safest
usage is by means of the \texttt{decorators.decorated()} function, which
takes an object (a function or a class) and checks its docstring: if 
a magic docstring is found, it returns a decorated version of the object, 
otherwise it returns the original object. Using \texttt{decorators.decorated()}
is simple but verbose, so magic shortcuts will be discussed in the next 
sections.

Here, let me give an example, showing that method decorators work both for 
\href{http://www.python.org/2.3/descrintro.html}{new style classes and old style classes}:
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{{\#}{$<$}example1.py{$>$}}\\
\mbox{}\\
\mbox{import~decorators}\\
\mbox{}\\
\mbox{def~do{\_}nothing(self):}\\
\mbox{~~~"No~magic~docstring~here"}\\
\mbox{dec{\_}do{\_}nothing=decorators.decorated(do{\_}nothing)}\\
\mbox{}\\
\mbox{def~identity(x):}\\
\mbox{~~~~"[staticmethod]"}\\
\mbox{~~~~return~x}\\
\mbox{dec{\_}identity=decorators.decorated(identity)~}\\
\mbox{}\\
\mbox{def~name(cls):}\\
\mbox{~~~~"[classmethod]"}\\
\mbox{~~~~return~cls.{\_}{\_}name{\_}{\_}}\\
\mbox{dec{\_}name=decorators.decorated(name)}\\
\mbox{}\\
\mbox{class~OldStyle:}\\
\mbox{~~~~do{\_}nothing=dec{\_}do{\_}nothing}\\
\mbox{~~~~identity=dec{\_}identity}\\
\mbox{}\\
\mbox{class~NewStyle(object):}\\
\mbox{~~~~name=dec{\_}name}\\
\mbox{}\\
\mbox{o=OldStyle()~{\#}~creates~an~old~style~instance}\\
\mbox{n=NewStyle()~{\#}~creates~a~new~style~instance}\\
\mbox{}\\
\mbox{{\#}{$<$}/example1.py{$>$}}
\end{flushleft}\end{ttfamily}
\end{quote}
\begin{verbatim}>>> from example1 import * # for testing purposes\end{verbatim}

In this example, both \texttt{dec{\_}identity} and \texttt{dec{\_}name} are decorator objects,
i.e. descriptors modifiying the attribute access. It is easy to recognize
decorators objects in the interpreter, since they have a re-defined 
printing representation:
\begin{verbatim}>>> print dec_identity
<staticmethod:identity>
>>> print dec_name
<classmethod:name>\end{verbatim}

On the other hand, \texttt{do{\_}nothing} does not have a magic 
docstring, therefore it is not converted to a decorator object; 
actually it is \emph{exactly} the original function
\begin{verbatim}>>> dec_do_nothing is do_nothing # not converted
True\end{verbatim}

and it works as a standard method:
\begin{verbatim}>>> o.do_nothing() # does nothing, correct\end{verbatim}

On the contrary, \texttt{dec{\_} identity} works as a staticmethod,
\begin{verbatim}>>> OldStyle.identity(1) # called from the class
1
>>> o.identity(1) # called from the instance
1\end{verbatim}

whereas \texttt{dec{\_}name} works as a classmethod:
\begin{verbatim}>>> NewStyle.name() # called from the class
'NewStyle'
>>> n.name() # called from the instance
'NewStyle'\end{verbatim}

Notice that, I have re-implemented the built-in 
\texttt{staticmethod} and \texttt{classmethod}, so
\begin{verbatim}>>> isinstance(dec_identity,staticmethod)
False\end{verbatim}

and
\begin{verbatim}>>> isinstance(dec_name,classmethod)
False\end{verbatim}

It is possible to recognize method decorators since they provides 
a couple of special attributes:
\begin{itemize}
\item 
\texttt{{\_}{\_}func{\_}{\_}}: returning the function from which they originated:
\begin{verbatim}>>> assert dec_identity.__func__ is identity
>>> assert dec_name.__func__ is name\end{verbatim}

\item 
\texttt{{\_}{\_}klass{\_}{\_}}: returning the class where they where defined:
\begin{verbatim}>>> dec_identity.__klass__
<class 'safetype.?'>\end{verbatim}

\end{itemize}

The question mark here means that the definition class is unknown.
The \texttt{{\_}{\_}klass{\_}{\_}} attribute can be set by hand or automatically
with the trick explained in the next section.


%___________________________________________________________________________

\hypertarget{decorating-classes}{}
\subsection*{Decorating classes}
\pdfbookmark[1]{Decorating classes}{decorating-classes}

The problem with the approach just described 
is that it does not present any significant advantage over 
the already existing mechanism. A real step forward would be to
have classes with the ability of automatically decorating their 
methods according to the docstrings. 
This sounds a bit of magic, but actually can be done very simply 
by adding to the class a docstring starting with ``[Decorated]''
and by decorating the class. 
Here is an example:
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{{\#}{$<$}example2.py{$>$}}\\
\mbox{}\\
\mbox{from~decorators~import~decorated}\\
\mbox{from~example1~import~do{\_}nothing,identity,name}\\
\mbox{}\\
\mbox{class~B(object):}\\
\mbox{~~~~"This~is~a~regular~class"}\\
\mbox{}\\
\mbox{B=decorated(B)~{\#}~does~nothing}\\
\mbox{}\\
\mbox{class~C(B):}\\
\mbox{~~~"[Decorated]"}\\
\mbox{~~~do{\_}nothing=do{\_}nothing}\\
\mbox{~~~identity=identity}\\
\mbox{~~~class~Inner:~{\#}~old~style~class}\\
\mbox{~~~~~~~"[Decorated]"~{\#}~required~docstring~~~}\\
\mbox{~~~~~~~name=name}\\
\mbox{}\\
\mbox{C=decorated(C)~{\#}~regenerates~the~class~converting~methods~in~decorators}\\
\mbox{c=C()}\\
\mbox{}\\
\mbox{{\#}{$<$}/example2.py{$>$}}
\end{flushleft}\end{ttfamily}
\end{quote}

Under the hood \texttt{decorators.decorated()} recognizes the class level
magic docstring ``[Decorated]'' and creates an instance of the 
\texttt{decorators.Decorated} metaclass.
Internally the metaclass invokes \texttt{decorators.decorated()}
on the methods of its instances: this is why they becomes decorated
if a suitable docstring is found. Moreover, it decorates inner classes,
if a suitable docstring is found, and it works recursively.

Here is the testing:
\begin{verbatim}>>> from example2 import *
>>> assert C.identity(1) == 1
>>> assert C.Inner.name() == 'Inner'
>>> assert c.identity(1) == 1 
>>> assert c.Inner.name() == 'Inner' \end{verbatim}

Notice that adding \texttt{identity} after the class creation with the syntax 
\texttt{C.identity=identity} would not work; 
\texttt{C.identity=decorators.decorated(identity)} is required:
\begin{verbatim}>>> C.identity=decorators.decorated(identity)
>>> C.identity(1) # it works
1\end{verbatim}

If a class misses the magic docstring, nothing happens:
\begin{verbatim}>>> B # returns the original B
<class 'example2.B'>\end{verbatim}

The mechanism works for old style classes too,
since the metaclass automagically converts the old style classes in a 
new style one:
\begin{verbatim}>>> type(C.Inner) # C.Inner is an instance of decorator.Decorated
<class 'decorators.Decorated'>\end{verbatim}

The enhancement provided by the metaclass includes a new default 
printing representation for both the class
\begin{verbatim}>>> print C # returns the name of D and of its metaclass
<class C[Decorated]>\end{verbatim}

and its instances:
\begin{verbatim}>>> print c
<C instance>\end{verbatim}

On the other hand, if a custom printing representation is already
defined, the metaclass takes it without any change.

One can even forget the docstring in subclasses of decorated 
classes, since metaclasses are inherited:
\begin{verbatim}>>> class D(C):
...     def name(cls):
...          "[classmethod]"
...          return cls.__name__
>>> print D.name()
D\end{verbatim}

Decorating whole classes presents another advantage: the decorated methods know
the class where they were defined via the special attribute \texttt{{\_}{\_}klass{\_}{\_}}:
\begin{verbatim}>>> D.__dict__['name'].__klass__ # the class where 'name' is defined
<class 'D'>\end{verbatim}

This is useful for introspection and debugging purposes.


%___________________________________________________________________________

\hypertarget{adding-magic}{}
\subsection*{Adding magic}
\pdfbookmark[1]{Adding magic}{adding-magic}

The problem of the previous approach is that one must explicitely
decorate the classes by hand, by invoking \texttt{decorators.decorated()}
each time. However, it is possible to add more magic
and to decorate all the classes automatically. 
It is as easy as writing \texttt{decorators.enhance{\_}classes()}
on top of you module. Then all methods in all classes with a magic docstring 
will be checked for magic docstrings and automagically decorated if needed. 
For instance, the previous example would be written
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{{\#}{$<$}example3.py{$>$}}\\
\mbox{}\\
\mbox{import~decorators;~decorators.enhance{\_}classes()}\\
\mbox{}\\
\mbox{class~C:}\\
\mbox{~~~~"[Decorated]"~{\#}~magic~docstring~here~}\\
\mbox{~~~~def~do{\_}nothing(self):}\\
\mbox{~~~~~~~"No~magic~docstring~here"}\\
\mbox{}\\
\mbox{~~~~def~identity(x):}\\
\mbox{~~~~~~~~"[staticmethod]"}\\
\mbox{~~~~~~~~return~x}\\
\mbox{}\\
\mbox{class~D(object):}\\
\mbox{~~~~"Undecorated"~{\#}~no~magic~docstring~here}\\
\mbox{~~~~def~name(cls):}\\
\mbox{~~~~~~~~"[classmethod]"}\\
\mbox{~~~~~~~~return~cls.{\_}{\_}name{\_}{\_}}\\
\mbox{}\\
\mbox{c=C();~d=D()}\\
\mbox{}\\
\mbox{{\#}{$<$}/example3.py{$>$}}
\end{flushleft}\end{ttfamily}
\end{quote}

\texttt{C} has a \texttt{[Decorated]} docstring, so its methods
are automatically decorated:
\begin{verbatim}>>> from example3 import *
>>> assert c.do_nothing() is None
>>> assert C.identity(1) == 1
>>> assert c.identity(1) == 1 \end{verbatim}

On the other hand, since \texttt{D} misses a magic docstring,
its \texttt{name} method is not decorated:
\begin{verbatim}>>> hasattr(D.__dict__['name'],'__func__') # not a decorator
False\end{verbatim}

Since \texttt{D.name} is a regular method and not a classmethod, \texttt{D.name()}
gives an error:
\begin{verbatim}>>> D.name()
Traceback (most recent call last):
  ...
TypeError: unbound method name() must be called with D instance 
as first argument (got nothing instead)\end{verbatim}

Under the hood, the magic works by enhancing the \texttt{object} class 
of the module with a \texttt{decorators.ClassDecorator} metaclass:
\begin{verbatim}>>> import example4
>>> type(example4.object)
<class 'decorators.ClassDecorator'>\end{verbatim}

Notice that for safety reasons the enhancement is only on the module 
\texttt{object} class, not on the \texttt{{\_}{\_}builtin{\_}{\_}.object} class. The dangers of
adding too much magic are discussed in the \href{#diving-into-magic}{Diving into magic} section.


%___________________________________________________________________________

\hypertarget{defining-method-decorators}{}
\subsection*{Defining method decorators}
\pdfbookmark[1]{Defining method decorators}{defining-method-decorators}

The \texttt{decorators} module contains two predefinite method decorators, 
\texttt{staticmethod} and \texttt{classmethod}, which emulate the built-ins 
with the same names. However, it is possible to write your own
custom decorators. The \texttt{decorators.MethodDecorator} class which is here 
exactly for that purpose.

Custom decorators are expected to be implemented by subclassing
\texttt{MethodDecorator} and by overriding its \texttt{get} method. The
\texttt{get} method automagically induces a \texttt{{\_}{\_}get{\_}{\_}} method, turning the 
class in a descriptor. The machinery is needed since \texttt{{\_}{\_}get{\_}{\_}} cannot 
be made cooperative using  the standard \texttt{super} mechanism because
there would be a confusion between \texttt{super.{\_}{\_}get{\_}{\_}} and the decorator 
\texttt{{\_}{\_}get{\_}{\_}}. This is a bit tricky, but the causal programmer is not 
expected to write custom decorators, and actually I don't want to make 
the access to decorators \emph{too} easy, since they are potentially dangerous.

In order to give a simple example, let me show the implementation 
of a \texttt{chattymethod} that prints a message when it is called:
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{{\#}{$<$}customdec.py{$>$}}\\
\mbox{}\\
\mbox{from~decorators~import~*~~}\\
\mbox{}\\
\mbox{class~chattymethod(MethodDecorator):}\\
\mbox{~~~~logfile=sys.stdout~{\#}~default}\\
\mbox{~~~~def~get(self,obj,cls=None):~{\#}~same~signature~as~{\_}{\_}get{\_}{\_}}\\
\mbox{~~~~~~~~self.logfile.write('calling~{\%}s~from~{\%}s{\textbackslash}n'~{\%}~(self,obj~or~cls))}\\
\mbox{~~~~~~~~return~super(chattymethod,self).get(obj,cls)}\\
\mbox{}\\
\mbox{{\#}{$<$}/customdec.py{$>$}}
\end{flushleft}\end{ttfamily}
\end{quote}

Notice the usage of the \texttt{super().get} trick. This guarantees that 
\texttt{chattymethod} will play well with other decorators (i.e. it
can be nicely composed via multiple inheritance). The point will
be fully discussed in the section on composing decorators.

Here is an example of usage:
\begin{verbatim}>>> import customdec # adds chattymethod to the list of known decorators
>>> customdec.enhance_classes() # automagically enhances classes
>>> class C: 
...     " [Decorated] "
...     def f(self): 
...         """
...         [ chattymethod ]
...         """
>>> c=C()
>>> c.f()
calling <chattymethod:f> from <C instance>\end{verbatim}

By the way, this shows that one can safely add whitespaces (including
newlines) to the magic docstring: they are simply ignored.

One can check that the syntax \texttt{C.f(c)} works too:
\begin{verbatim}>>> C.f(c) 
calling <chattymethod:f> from <class C[Decorated]>\end{verbatim}

A tricky point of the decorators mechanism is the issue of parameter passing.
In comp.lang.python there was the proposal of allowing explicit parameter
passing to decorators, with a syntax of kind
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{def~f(self)[chattymethod(logfile=file('file1.log','w'))]}
\end{flushleft}\end{ttfamily}
\end{quote}

In my view, there are too many parenthesis in this syntax, and it 
becomes rapidly unreadable. Moreover, it complicates the implementation
without any real benefit, so the \texttt{decorators} module does not allow
this kind of parameter passings. There are however viable
workarounds, so you should not miss the syntax.

A simple minded solution is to change the defaults by hand:
\begin{verbatim}>>> from customdec import chattymethod,decorated
>>> chattymethod.logfile=file('file.log','w')
>>> def g(self): 
...     "[chattymethod]"
>>> C.g=decorated(g)
>>> c.g() # will print a message on file.log\end{verbatim}

This approach has the drawback that chattymethods created before changing 
the logfile will also print to the new logfile, if invoked after the
change. Therefore
\begin{verbatim}>>> c.f()\end{verbatim}

will print a message to \texttt{file.log} too, and not to standard output.
Here is the confirmation:
\begin{verbatim}>>> chattymethod.logfile.close()
>>> print file('file.log').read().rstrip()
calling <chattymethod:g> from <C instance>
calling <chattymethod:f> from <C instance>\end{verbatim}

A better solution is to pass  
parameters to method decorators as function attributes: then the function
attributes can be converted to attributes of the decorator 
in  the \texttt{{\_}{\_}init{\_}{\_}} method. Here is an example:
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{{\#}{$<$}customdec.py{$>$}}\\
\mbox{}\\
\mbox{class~chattymethod2(chattymethod):~}\\
\mbox{~~~~logfile=sys.stdout~{\#}~default}\\
\mbox{~~~~def~{\_}{\_}init{\_}{\_}(self,objfunc):}\\
\mbox{~~~~~~~~super(chattymethod2,self).{\_}{\_}init{\_}{\_}(objfunc)}\\
\mbox{~~~~~~~~logfile=getattr(self.{\_}{\_}func{\_}{\_},'logfile',None)}\\
\mbox{~~~~~~~~if~logfile:~self.logfile=logfile~}\\
\mbox{}\\
\mbox{{\#}{$<$}/customdec.py{$>$}}
\end{flushleft}\end{ttfamily}
\end{quote}

Notice that the \texttt{{\_}{\_}init{\_}{\_}} method has the signature
\texttt{{\_}{\_}init{\_}{\_}(self,objfunc)}, where the \texttt{objfunc} object is a
decorator object or the function to be converted in the decorator 
object, and that it is cooperative.
This is the suggested way of overriding \texttt{{\_}{\_}init{\_}{\_}}.

Here is the testing:
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{{\#}{$<$}chatty2.py{$>$}}\\
\mbox{}\\
\mbox{import~customdec;~customdec.enhance{\_}classes()}\\
\mbox{}\\
\mbox{{\#}~sets~the~log~files}\\
\mbox{log1=file('file1.log','w')}\\
\mbox{log2=file('file2.log','w')}\\
\mbox{}\\
\mbox{class~C:}\\
\mbox{~~~~"[Decorated]"}\\
\mbox{~~~~def~f(self):~}\\
\mbox{~~~~~~~~"[chattymethod2]"}\\
\mbox{~~~~f.logfile=log1~{\#}~function~attribute}\\
\mbox{~~~~def~g(self):~}\\
\mbox{~~~~~~~~"[chattymethod2]"}\\
\mbox{~~~~g.logfile=log2~{\#}~function~attribute}\\
\mbox{}\\
\mbox{assert~C.{\_}{\_}dict{\_}{\_}['f'].logfile~is~log1~{\#}~check~the~conversion~}\\
\mbox{assert~C.{\_}{\_}dict{\_}{\_}['g'].logfile~is~log2~{\#}~function~attr.~-{$>$}~decorator~attr.}\\
\mbox{}\\
\mbox{c=C()~{\#}~C~instantiation}\\
\mbox{}\\
\mbox{c.f()~{\#}~print~a~message~in~file1.log}\\
\mbox{c.g()~{\#}~print~a~message~in~file2.log}\\
\mbox{}\\
\mbox{log1.close();~log2.close()~{\#}~finally}\\
\mbox{}\\
\mbox{{\#}{$<$}/chatty2.py{$>$}}
\end{flushleft}\end{ttfamily}
\end{quote}

Let me check the contents of the log files:
\begin{verbatim}>>> import chatty2
>>> print file('file1.log').read().rstrip()
calling <chattymethod2:f> from <C instance>
>>> print file('file2.log').read().rstrip()
calling <chattymethod2:g> from <C instance>\end{verbatim}

\texttt{chattymethod} is the poor man version of \texttt{tracedmethod}, a
sophisticated decorator for tracing methods.
Here is the code, given for pedagogical purposes; the lazy reader can 
skip it and go directly to the usage section.
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{{\#}{$<$}customdec.py{$>$}}\\
\mbox{}\\
\mbox{class~tracedmethod(MethodDecorator):}\\
\mbox{~~~~"Descriptor~class,~converts~a~method~in~a~traced~method"}\\
\mbox{~~~~indent=0;~output=sys.stdout~{\#}~defaults}\\
\mbox{}\\
\mbox{~~~~def~{\_}{\_}init{\_}{\_}(self,objfunc):}\\
\mbox{~~~~~~~~super(tracedmethod,self).{\_}{\_}init{\_}{\_}(objfunc)}\\
\mbox{~~~~~~~~self.funcname=self.{\_}{\_}func{\_}{\_}.{\_}{\_}name{\_}{\_}}\\
\mbox{~~~~~~~~output=getattr(self.{\_}{\_}func{\_}{\_},'output',None)~}\\
\mbox{~~~~~~~~if~output:~self.output=output~{\#}~func.attr.~-{$>$}~dec.attr.}\\
\mbox{}\\
\mbox{~~~~def~get(self,obj,cls):}\\
\mbox{~~~~~~~~clsname=self.{\_}{\_}klass{\_}{\_}.{\_}{\_}name{\_}{\_}~{\#}~definition~clas}\\
\mbox{~~~~~~~~def~tracedmeth(obj,*args,**kw):}\\
\mbox{~~~~~~~~~~~~i='~'*self.indent~{\#}~default~indentation}\\
\mbox{~~~~~~~~~~~~self.{\_}{\_}class{\_}{\_}.indent+=4~{\#}~increases~indentation}\\
\mbox{~~~~~~~~~~~~self.output.write("{\%}sCalling~'{\%}s.{\%}s'~with~arguments~"~{\%}~}\\
\mbox{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~(i,clsname,self.funcname))}\\
\mbox{~~~~~~~~~~~~self.output.write("{\%}s{\%}s~...{\textbackslash}n"~{\%}~(obj~or~'',str(args)+str(kw)))}\\
\mbox{~~~~~~~~~~~~res=super(tracedmethod,self).get(obj,cls)(*args,**kw)}\\
\mbox{~~~~~~~~~~~~self.output.write("{\%}s'{\%}s.{\%}s'~called~with~result:~{\%}s{\textbackslash}n"}\\
\mbox{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{\%}~(i,clsname,self.funcname,res))}\\
\mbox{~~~~~~~~~~~~self.{\_}{\_}class{\_}{\_}.indent-=4~{\#}~restores~default~indentation}\\
\mbox{~~~~~~~~~~~~return~res}\\
\mbox{~~~~~~~~return~tracedmeth.{\_}{\_}get{\_}{\_}(obj,cls)~{\#}~method~wrapper}\\
\mbox{}\\
\mbox{{\#}{$<$}/customdec.py{$>$}}
\end{flushleft}\end{ttfamily}
\end{quote}

\texttt{tracedmethod.get} returns a method wrapper object, so it is
possible to use \texttt{im{\_}func} to retrieve the internal function
\texttt{tracedmeth}:
\begin{verbatim}>>> class C:
...     "[Decorated]"
...     def f(self):
...         "[tracedmethod]"
>>> c=C(); c.f() 
Calling 'C.f' with arguments <C instance>(){} ...
'C.f' called with result: None
>>> c.f.im_func.__name__
'tracedmeth'\end{verbatim}

As soon as the \texttt{tracedmethod} module is loaded, the \texttt{tracedmethod} class
is added to the list of know decorators, so one should use the  
``[tracedmethod]'' docstring and not something like
``[customdec.tracedmethod]''.

Here is a less trivial example of usage, writing in a log file the 
internal working of a recursive function:
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{{\#}{$<$}example9.py{$>$}}\\
\mbox{}\\
\mbox{import~customdec;~customdec.enhance{\_}classes()}\\
\mbox{}\\
\mbox{logfile=file('file3.log','w')}\\
\mbox{}\\
\mbox{class~C(object):}\\
\mbox{~~~~"[Decorated]"}\\
\mbox{~~~~def~fact(self,n):}\\
\mbox{~~~~~~~~"[tracedmethod]~The~good~old~factorial."}\\
\mbox{~~~~~~~~if~n==0:~return~1}\\
\mbox{~~~~~~~~else:~return~n*self.fact(n-1)}\\
\mbox{~~~~fact.output=logfile}\\
\mbox{}\\
\mbox{C().fact(2)~{\#}~write~a~message~to~logfile}\\
\mbox{}\\
\mbox{logfile.close()}\\
\mbox{}\\
\mbox{{\#}{$<$}/example9.py{$>$}}
\end{flushleft}\end{ttfamily}
\end{quote}

Here is the content of \texttt{file3.log}:
\begin{verbatim}>>> import example9 # creates file3.log
>>> print file('file3.log').read().rstrip()
Calling 'C.fact' with arguments <C instance>(2,){} ...
    Calling 'C.fact' with arguments <C instance>(1,){} ...
        Calling 'C.fact' with arguments <C instance>(0,){} ...
        'C.fact' called with result: 1
    'C.fact' called with result: 1
'C.fact' called with result: 2\end{verbatim}


%___________________________________________________________________________

\hypertarget{defining-class-decorators}{}
\subsection*{Defining class decorators}
\pdfbookmark[1]{Defining class decorators}{defining-class-decorators}

PEP 318 proposes to decorate methods by using descriptors; it is
quite natural to extend this idea and to decorate classes by using 
class decorators implemented as metaclasses. We already saw a couple of
class decorator at work, the metaclass \texttt{ClassDecorator}, which gives
to its instances the ability to interpret class level magic docstrings, and
its subclass \texttt{Decorated}, which converts functions in method decorators
if suitable docstrings are found.

To define a custom class decorator is easy: one defines a custom metaclass 
as usual, with the only difference of deriving from \texttt{ClassDecorator} instead
of deriving from \texttt{type}.

To understand how this works in practice, let me 
show how to add logging capabilities to a given class. The first
step is to define a suitable class decorator, such as the following:
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{{\#}{$<$}customdec.py{$>$}}\\
\mbox{}\\
\mbox{class~Logged(ClassDecorator):}\\
\mbox{~~~~output=sys.stdout}\\
\mbox{~~~~def~{\_}{\_}init{\_}{\_}(cls,name,bases,dic):}\\
\mbox{~~~~~~~~super(Logged,cls).{\_}{\_}init{\_}{\_}(name,bases,dic)}\\
\mbox{~~~~~~~~print~{$>$}{$>$}~cls.output,"{\%}s~created"~{\%}~cls}\\
\mbox{}\\
\mbox{{\#}{$<$}/customdec.py{$>$}}
\end{flushleft}\end{ttfamily}
\end{quote}

\texttt{Logged} is derived by the metaclass \texttt{ClassDecorator}, 
which provides a certain amount of magic under the hood (in particular
its printing representation and its calling syntax are redefined by its
(meta-)metaclass \texttt{MetaDecorator}). 
Logging capabilities can be added to a class 
by simply using the magic docstring syntax:
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{{\#}{$<$}logged.py{$>$}}\\
\mbox{}\\
\mbox{import~customdec;~customdec.enhance{\_}classes()}\\
\mbox{}\\
\mbox{class~D(object):~{\#}~will~print~a~message~at~D~creation}\\
\mbox{~~~~"[Logged]"}\\
\mbox{}\\
\mbox{{\#}{$<$}/logged.py{$>$}}
\end{flushleft}\end{ttfamily}
\end{quote}
\begin{verbatim}>>> import logged
<class 'logged.D'> created\end{verbatim}

Notice that the printing representation of \texttt{D} involves the name
of \texttt{D} preceded by the name of its metaclass, which in this case
is \texttt{Logged}

Each time an instance of \texttt{Logged} is created, a similar message is printed:
\begin{verbatim}>>> class E(logged.D):
...     pass
<class 'E'> created\end{verbatim}

Notice that \texttt{E} does not have any magic docstring
\begin{verbatim}>>> E.__doc__ # no docstring\end{verbatim}

but still it inherits its magic from \texttt{D}.

Another simple example of class decorator is the following metaclass 
which modifies the docstrings of the methods of its instances, 
by magically inducing tracing capabilities on them:
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{{\#}{$<$}customdec.py{$>$}}\\
\mbox{}\\
\mbox{from~types~import~FunctionType~~}\\
\mbox{}\\
\mbox{class~Traced(Decorated):}\\
\mbox{~~~~def~{\_}{\_}init{\_}{\_}(cls,n,b,d):}\\
\mbox{~~~~~~~~for~name,func~in~d.iteritems():}\\
\mbox{~~~~~~~~~~~~if~isinstance(func,FunctionType)~and~name!='{\_}{\_}str{\_}{\_}':~}\\
\mbox{~~~~~~~~~~~~~~~~{\#}~cannot~trace~{\_}{\_}str{\_}{\_},~since~it~is~invoked~by}\\
\mbox{~~~~~~~~~~~~~~~~{\#}~tracedmethod~and~would~generate~infinite~recursion}\\
\mbox{~~~~~~~~~~~~~~~~func.{\_}{\_}doc{\_}{\_}="[tracedmethod]~"~+~(func.{\_}{\_}doc{\_}{\_}~or~'')}\\
\mbox{~~~~~~~~super(Traced,cls).{\_}{\_}init{\_}{\_}(n,b,d)~{\#}~decorates~the~methods}\\
\mbox{}\\
\mbox{{\#}{$<$}/customdec.py{$>$}}
\end{flushleft}\end{ttfamily}
\end{quote}

Here is an example of usage:
\begin{verbatim}>>> class C(object):
...     """[Traced] The class decorator adds the magic docstring
...     '[tracedmethod]' to f1 and f2, which are then converted 
...     in method decorator objects."""
...     def f1(self): pass
...     def f2(self): pass
...
>>> type(C)
<class 'customdec.Traced'>
>>> c=C()
>>> c.f1()
Calling 'C.f1' with arguments <C instance>(){} ...
'C.f1' called with result: None
>>> c.f2()
Calling 'C.f2' with arguments <C instance>(){} ...
'C.f2' called with result: None\end{verbatim}

By default, the decorators module only decorates classes with a magic 
docstring (and they subclasses, even without magic docstrings). 
If all the classes of your module have the same magic docstring, 
it makes sense to decorate them all
with a single command. It is enough to use \texttt{decorators.enhance{\_}classes()}
with a magic docstring corresponding to a class decorator as argument, 
as in this example:
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{{\#}{$<$}example7.py{$>$}}\\
\mbox{}\\
\mbox{from~example2~import~identity,name}\\
\mbox{import~inspect,~decorators;~decorators.enhance{\_}classes("[Decorated]")}\\
\mbox{}\\
\mbox{class~C1(object):~{\#}~automagically~converted~to~a~decorated~class}\\
\mbox{~~~~identity=identity~}\\
\mbox{}\\
\mbox{class~C2:~{\#}~automagically~converted~to~a~decorated~class}\\
\mbox{~~~~name=name}\\
\mbox{}\\
\mbox{c1=C1()~{\#}~C1~instance}\\
\mbox{c2=C2()~{\#}~C2~instance}\\
\mbox{}\\
\mbox{{\#}{$<$}/example7.py{$>$}}
\end{flushleft}\end{ttfamily}
\end{quote}

The magic works both for new style classes such as \texttt{C1}
\begin{verbatim}>>> from example7 import C1,c1
>>> assert C1.identity(1) == 1
>>> assert c1.identity(1) == 1 \end{verbatim}

and old style classes such as \texttt{C2} by implicitly converting them
to new style classes:
\begin{verbatim}>>> from example7 import C2,c2 
>>> assert C2.name() == 'C2'
>>> assert c2.name() == 'C2' \end{verbatim}


%___________________________________________________________________________

\hypertarget{composing-decorators}{}
\subsection*{Composing decorators}
\pdfbookmark[1]{Composing decorators}{composing-decorators}

Decorators can be composed by using magic docstrings with comma-separated
decorator names. For instance, you can trace a classmethod:
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{{\#}{$<$}example6.py{$>$}}\\
\mbox{}\\
\mbox{"How~to~trace~a~class~method"}\\
\mbox{}\\
\mbox{import~customdec;~customdec.enhance{\_}classes()}\\
\mbox{}\\
\mbox{class~C(object):}\\
\mbox{~~~~"[Decorated]"}\\
\mbox{~~~~def~fact(cls,n):~{\#}~a~traced~classmethod}\\
\mbox{~~~~~~~~"[classmethod,tracedmethod]"}\\
\mbox{~~~~~~~~if~n==0:~return~1}\\
\mbox{~~~~~~~~else:~return~n*cls.fact(n-1)}\\
\mbox{}\\
\mbox{{\#}{$<$}/example6.py{$>$}}
\end{flushleft}\end{ttfamily}
\end{quote}

Here is the testing:
\begin{verbatim}>>> from example6 import C
>>> C.fact(2)
Calling 'C.fact' with arguments <class C[Decorated]>(2,){} ...
    Calling 'C.fact' with arguments <class C[Decorated]>(1,){} ...
        Calling 'C.fact' with arguments <class C[Decorated]>(0,){} ...
        'C.fact' called with result: 1
    'C.fact' called with result: 1
'C.fact' called with result: 2
2\end{verbatim}

You may easily check that calling \texttt{C().fact} will work too.

Under the hood the syntax
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{[classmethod,tracedmethod]}
\end{flushleft}\end{ttfamily}
\end{quote}

generates a \texttt{classmethodtracedmethod} class obtained via
multiple inheritance:
\begin{verbatim}>>> type(C.__dict__['fact'])
<class 'safetype.classmethodtracedmethod'>\end{verbatim}

Notice that the order \emph{does} matter and using the docstring 
``[tracedmethod,classmethod]'' will not work:
\begin{verbatim}>>> class D:
...     "[Decorated]"
...     def f(cls):
...         "[tracedmethod,classmethod]"
>>> D.f()
Traceback (most recent call last):
  ...
TypeError: unbound method tracedmeth() must be called with D instance 
as first argument (got nothing instead)\end{verbatim}

The problem here is that \texttt{tracedmethod.get} returns a method-wrapper object
which expects a D instance 
as first argument whereas it gets \texttt{None} since
it is called from the class. On the other hand,
\begin{verbatim}>>> D().f()
Calling 'D.f' with arguments <D instance>(){} ...
'D.f' called with result: None\end{verbatim}

will work. When \texttt{classmethod} precedes \texttt{tracedmethod}, then
\texttt{classmethod} passes to \texttt{tracedmeth} a non-empty first argument,
i.e. the calling class, even when called from the instance:
\begin{verbatim}>>> C().fact(2)
Calling 'C.fact' with arguments <class C[Decorated]>(2,){} ...
    Calling 'C.fact' with arguments <class C[Decorated]>(1,){} ...
        Calling 'C.fact' with arguments <class C[Decorated]>(0,){} ...
        'C.fact' called with result: 1
    'C.fact' called with result: 1
'C.fact' called with result: 2
2\end{verbatim}

If we try to trace a staticmethod, we will get a different error with
the order ``tracedmethod, staticmethod'':
\begin{verbatim}>>> class F(object):
...    "[Decorated]"
...    def fact(n):
...        "[tracedmethod,staticmethod]"
...        if n==0: return 1
...        else: return n*F.fact(n-1)
>>> F.fact(2)
Traceback (most recent call last):
  ...
TypeError: unbound method tracedmeth() must be called with F instance 
as first argument (got int instance instead)\end{verbatim}

The message is self-explanatory.

On the other hand, composing the decorators in the order
``[tracedmethod,staticmethod]'' will work just fine.

It is possible to compose class decorators just as method decorators,
by using the docstring syntax: for instance we may create a class
which is both \texttt{Decorated} and \texttt{Logged} and by trivially writing
\begin{verbatim}>>> class C: 
...     "[Decorated,Logged]"
...     def f(): 
...         "[staticmethod]"
...         return 'it works!'
<class C[DecoratedLogged]> created
>>> C().f()
'it works!'\end{verbatim}


%___________________________________________________________________________

\hypertarget{diving-into-magic}{}
\subsection*{Diving into magic}
\pdfbookmark[1]{Diving into magic}{diving-into-magic}

If you never use metaclasses, this part can be safely skipped. On the
other hand, if you are a metaclass user, you \emph{must} read this section
in order to keep your code working.

The \texttt{decorators} module provide a \texttt{ClassDecorator} metaclass which 
converts a regular (both old style or new style) class in a class with 
the ability to recognize magic docstrings. Under the hood, the 
\texttt{decorators.enhance{\_}classes()} trick works by decorating the 
\texttt{object} class with \texttt{decorators.ClassDecorator} and by setting 
the custom metaclass to \texttt{decorators.ClassDecorator} and it is 
equivalent to
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{import~decorators}\\
\mbox{object=decorators.ClassDecorator(object)~{\#}~decorates~new~style~classes}\\
\mbox{{\_}{\_}metaclass{\_}{\_}=~decorators.ClassDecorator~{\#}~decorates~old~style~classes}
\end{flushleft}\end{ttfamily}
\end{quote}

If you want the magic to work only for new style classes only, you may 
forget the second line; if you want the magic to work for old style 
classes only, you may forget the first line.

The \texttt{decorators.enhance{\_}classes("[SomeClassDec]")} syntax looks at the
magic docstring and executes something like
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{import~decorators}\\
\mbox{object=decorators.SomeClassDec(object)~{\#}~decorates~all~new~style~classes}\\
\mbox{{\_}{\_}metaclass{\_}{\_}=~decorators.SomeClassDec~{\#}~decorates~all~old~style~classes}
\end{flushleft}\end{ttfamily}
\end{quote}

The problem with the \texttt{enhance{\_}classes()} syntaxes is that it is
not 100{\%} safe under metaclass conflicts. In order to explain the issue,
let me give an example.

Suppose we enhance the \texttt{object} class in the interpreter namespace:
\begin{verbatim}>>> import decorators; decorators.enhance_classes()\end{verbatim}

making it an instance of \texttt{ClassDecorator}:
\begin{verbatim}>>> object.__class__ 
<class 'decorators.ClassDecorator'>\end{verbatim}

Now, if we naively create a custom metaclass \texttt{M}:
\begin{verbatim}>>> class M(type): 
...     "Some non-trivial code here..." \end{verbatim}

and we try to use it to enhance a new style class \texttt{NwithM}, we will get a
conflict:
\begin{verbatim}>>> class NwithM(object): __metaclass__=M # does not work!
...
Traceback (most recent call last):
  ...
TypeError: metaclass conflict: the metaclass of a derived class must be a 
(non-strict) subclass of the metaclasses of all its bases\end{verbatim}

The problem is that the previous line tries to create a class \texttt{NwithM} 
which should have both metaclasses \texttt{ClassDecorator} and \texttt{M}: 
a conflict follows.

Fortunately, the decorators module imports the \texttt{type} metaclass from
my \texttt{safetype} module just to avoid this kind of problems. If we
derive \texttt{M} from \texttt{decorators.type} (which is really \texttt{safetype.type})
the conflict is automagically avoided:
\begin{verbatim}>>> class M(decorators.type):
...     "This metaclass is conflict-proof"
>>> class NwithM(object): # it works!
...     __metaclass__=M\end{verbatim}

The reason why the conflict is avoided is that the \texttt{safetype} module
(which makes use of really \emph{deep} metaclass magic) automatically
creates the composed class \texttt{MClassDecorator} by multiply inheriting
from \texttt{M} and from \texttt{ClassDecorator}. Then, \texttt{NwithM} can be safely 
created as an instance of \texttt{safetype.MClassDecorator}:
\begin{verbatim}>>> type(NwithM)
<class 'safetype.MClassDecorator'>\end{verbatim}

The situation with old style classes is worse since
apparently
\begin{verbatim}>>> class OwithM: # old style class with metaclass M
...     __metaclass__=M
...     def sm(): 
...         "[staticmethod]"\end{verbatim}

gives no error, but actually \texttt{M} overrides 
\texttt{ClassDecorator}, so \texttt{OwithM} will not recognize magic docstrings:
\begin{verbatim}>>> type(OwithM)
<class 'M'>
>>> OwithM.sm()
Traceback (most recent call last):
  ...
TypeError: unbound method sm() must be called with OwithM instance 
as first argument (got nothing instead)\end{verbatim}

The simpler solution is to convert \texttt{OwithM} in a new style class:
\begin{verbatim}>>> class NwithM(OwithM,object):
...     __metaclass__=M
...     def sm():
...         "[staticmethod]"
>>> type(NwithM) 
<class 'safetype.MClassDecorator'>\end{verbatim}

Now \texttt{NwithM} is not decorated since it does miss a magic docstring, but
it provides the ability to recognizing magic docstrings, so \texttt{NwithM}
subclasses with a ``[Decorated]'' docstring will be decorated:
\begin{verbatim}>>> class E(NwithM): 
...     "[Decorated]"
...     def cm(cls):
...         "[classmethod]"
...         print '.cm() called from',cls\end{verbatim}
\begin{verbatim}>>> E.cm() # it works
.cm() called from <class E[MClassDecoratorDecorated]>\end{verbatim}

Notice that \texttt{sm} was defined in \texttt{NwithM}, the undecorated class: therefore
it is not decorated:
\begin{verbatim}>>> E.sm() # correctly, does not work
Traceback (most recent call last):
  ...
TypeError: unbound method sm() must be called with E instance 
as first argument (got nothing instead)\end{verbatim}

The error message says that \texttt{sm} is an unbound method and not a 
static method.

It is possible to go even deeper in \textbf{black} magic, and to decorate all 
the new style classes in \emph{all} modules, by decorating the 
\texttt{{\_}{\_}builtin{\_}{\_}.object}. Still, naively redefining the \texttt{{\_}{\_}builtin{\_}{\_}object} 
class is risky, since it will induce metaclass conflicts in other modules 
using metaclasses. In other words, it will work only if you import modules 
not using metaclasses, or modules using metaclasses safely, i.e. modules 
taking care of possible conflicts by using \texttt{safetype.type} as base metaclass.
If you really enjoy black magic, you may solve the problem by 
redefining the \texttt{{\_}{\_}builtin{\_}{\_}.type} as  \texttt{safetype.type}.
The \texttt{decorators} module does not go so deep in dark magic, but still
there are cases where you may want to do it. In these cases you must be
explicit and redefine the builtins by hand, without help from the
module. For instance, one of my original motivation for the usage
of metaclasses/class decorators was to use them for debugging purposes.
For instance, I wanted to trace the execution of methods in
complicate inheritance hierarchies, \emph{without changing the source code}.
For simple situations, i.e. in absence of inheritance and of pre-existing
metaclasses, the function \texttt{import{\_}with{\_}metaclass} discussed in the
\href{http://www-106.ibm.com/developerworks/library/l-pymeta.html}{first paper on metaclasses} written in collaboration with David Mertz, works 
fine but in general the implementation of a working \texttt{import{\_}with{\_}metaclass}
is cumbersome. For debugging purposes, the quick and dirty way can be a
good enough way, so let me show how we can redefine the builtins \texttt{object} and
\texttt{type} \emph{before} importing the module, in such a way to enhance \emph{all}
its classes with tracing capabilities.

Let me start from a module using an unsafe metaclass \texttt{M}, such that it
cannot be traced trivially by decorating its classes \emph{after} the import;
moreover there is an inheritance hierarchy, such that it cannot be
traced correctly by naively decorating all the classes one after the
other (unless one modifies the original source code, but this forbidden
by the rules of the game ;)
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{{\#}{$<$}tracing.py{$>$}}\\
\mbox{}\\
\mbox{"""}\\
\mbox{This~is~a~pre-existing~module~not~using~decorators~but~using~multiple}\\
\mbox{inheritance~and~unsafe~metaclasses.~We~want~to~trace~it~for~debugging~}\\
\mbox{purposes.}\\
\mbox{"""}\\
\mbox{~}\\
\mbox{class~M(type):~}\\
\mbox{~~~~"There~should~be~some~non-trivial~code~here"}\\
\mbox{}\\
\mbox{class~B(object):~}\\
\mbox{~~~~def~{\_}{\_}init{\_}{\_}(self):}\\
\mbox{~~~~~~~~super(B,self).{\_}{\_}init{\_}{\_}()}\\
\mbox{}\\
\mbox{class~D(object):~}\\
\mbox{~~~~{\_}{\_}metaclass{\_}{\_}=M}\\
\mbox{~~~~def~{\_}{\_}init{\_}{\_}(self):}\\
\mbox{~~~~~~~~super(D,self).{\_}{\_}init{\_}{\_}()}\\
\mbox{}\\
\mbox{class~E(B,D):}\\
\mbox{~~~~def~{\_}{\_}init{\_}{\_}(self):}\\
\mbox{~~~~~~~~super(E,self).{\_}{\_}init{\_}{\_}()}\\
\mbox{}\\
\mbox{~{\#}{$<$}/tracing.py{$>$}}
\end{flushleft}\end{ttfamily}
\end{quote}

Everything works is we modify the builtins before importing the module:
\begin{verbatim}>>> import __builtin__
>>> __object__=__builtin__.object # the original 'object' class
>>> __builtin__.object=customdec.Traced('tracedobject',(),{}) 
>>> __builtin__.type=customdec.type # a safe 'type' class\end{verbatim}

Now, when we import the module, all the classes \texttt{M}, \texttt{B}, \texttt{D} and \texttt{E}
will be created starting for the tricked builtins, so they will be traced
and safe under conflicts. For instance, let me show that \texttt{E} is created
with a conflict safe \texttt{MTraced} metaclass:
\begin{verbatim}>>> from tracing import E
>>> print E
<class E[MTraced]>\end{verbatim}

This shows that the \texttt{{\_}{\_}init{\_}{\_}} methods are traced indeed and shows the
Method Resolution Order:
\begin{verbatim}>>> e=E()
Calling 'E.__init__' with arguments <E instance>(){} ...
    Calling 'B.__init__' with arguments <E instance>(){} ...
        Calling 'D.__init__' with arguments <E instance>(){} ...
        'D.__init__' called with result: None
    'B.__init__' called with result: None
'E.__init__' called with result: None\end{verbatim}

This works, indeed. At the end, one should not forget to restore
the standard builtins, otherwise it will trace \emph{all} the classes
created thereafter.
\begin{verbatim}>>> __builtin__.object=__object__ # restore the __builtin__
>>> __builtin__.type=decorators.__type__ # restore the __builtin__\end{verbatim}

At the end, I will notice that it is possible to solve the problem more
nicely, without redefining the builtins, but I will discuss the solution
elsewhere ;)


%___________________________________________________________________________

\hypertarget{advanced-usage}{}
\subsection*{Advanced usage}
\pdfbookmark[1]{Advanced usage}{advanced-usage}

Whereas the average programmer is expected to use the \texttt{decorated()} 
and \texttt{enhance{\_}classes()} functions only, the \texttt{decorators} module provides 
facilities which may be useful to the advanced programmer.

The simplest is an utility function which retrieves the list of
recognized decorators, \texttt{decorators.getdec()}. The precise syntax is
\texttt{decorators.getdec(docstring)}, where \texttt{docstring}
is a magic docstring, i.e. a bracketed comma-separated list 
of decorator names. For instance \texttt{decorators.getdec('[MethodDecorator]')}
gives the list of all subclasses of \texttt{MethodDecorator}, i.e. all method
decorators, whereas \texttt{decorators.getdec('[ClassDecorator]')}
gives the list of the known class decorators. The utility of the function
is that it also returns decorators that have been automagically created:
\begin{verbatim}>>> decorators.getdec("[classmethodtracedmethod]")
[<class 'safetype.classmethodtracedmethod'>]\end{verbatim}

It is even possible to use the comma notation:
\begin{verbatim}>>> decorators.getdec("[classmethod,tracedmethod]")
[<class 'safetype.classmethodtracedmethod'>]\end{verbatim}

If you mispell a decorator name you get an helpful error message:
\begin{verbatim}>>> class E:
...     "[Decorated]"
...     def f():
...         "[staticmeth]"
Traceback (most recent call last):
  .. a cryptic traceback here ..
UnknownDecoratorError: staticmeth\end{verbatim}

By design, the \texttt{decorated} function is quite limited, and does not 
decorate functions without magic docstrings. This is safe and also convenient 
for internal usage of the \texttt{decorated} function in the \texttt{Decorated} 
metaclass. Nevertheless, advanced users may want to have the ability 
of decorating functions by hand, without invoking decorators.decorated()`
and magic docstrings. This is possible, by calling the decorator directly.
For instance, here is how to convert a lambda function in a staticmethod:
\begin{verbatim}>>> do_nothing=decorators.staticmethod(lambda:None)
>>> print do_nothing # ``do_nothing`` is a static method
<staticmethod:<lambda>>\end{verbatim}

The most convenient usage of this feature is in the composition of decorators. 
For instance, we may compose the just defined \texttt{staticmethod} with a 
\texttt{chattymethod2}:
\begin{verbatim}>>> chattystatic=customdec.chattymethod2(do_nothing)
>>> print chattystatic
<chattymethod2staticmethod:<lambda>>\end{verbatim}

The syntax
\begin{quote}

\texttt{decorator1(decorator2(obj))}
\end{quote}

automagically creates a composed class \texttt{decorator1decorator2} in the
\texttt{safetype} module (or recycle it, if \texttt{decorator1decorator2} has 
been already created) and it is equivalent to
\begin{quote}

\texttt{decorator1decorator2(obj)}
\end{quote}

Here is the check that everything works:
\begin{verbatim}>>> class B: 
...     chattystatic=chattystatic
>>> B.chattystatic()
calling <chattymethod2staticmethod:<lambda>> from <class 'B'>\end{verbatim}

Notice that \texttt{chattystatic} does not know the class where it 
is defined:
\begin{verbatim}>>> chattystatic.__klass__
<class 'safetype.?'>\end{verbatim}

unless the \texttt{{\_}{\_}klass{\_}{\_}} attribute is explicitly set.

This is the hierarchy:
\begin{verbatim}>>> for C in type(chattystatic).mro(): print C
...
<class 'safetype.chattymethod2staticmethod'>
<class 'customdec.chattymethod2'>
<class 'customdec.chattymethod'>
<class 'decorators.staticmethod'>
<class 'decorators.MethodDecorator'>
<class 'decorators.Decorator'>
<type 'object'>\end{verbatim}

One can also compose classes by hand, by using class decorators:
\begin{verbatim}>>> class C: 
...     "[Logged]"
...     def f(): "[staticmethod]"
...
<class 'C'> created
>>> C=customdec.Traced(C)
<class C[TracedLogged]> created\end{verbatim}

The \texttt{decorators.enhance{\_}classes({$<$}classdecorator{$>$})} syntax performs
the composition automagically:
\begin{quote}
\begin{ttfamily}\begin{flushleft}
\mbox{{\#}{$<$}example8.py{$>$}}\\
\mbox{}\\
\mbox{from~example2~import~identity,name}\\
\mbox{import~inspect,~decorators;~decorators.enhance{\_}classes("[Decorated]")}\\
\mbox{}\\
\mbox{class~C1:~{\#}~automagically~converted~to~a~Decorated~class}\\
\mbox{~~~~identity=identity~}\\
\mbox{}\\
\mbox{class~C2:~{\#}~automagically~converted~to~a~DecoratedLogged~class}\\
\mbox{~~~~"[Logged]"}\\
\mbox{~~~~name=name}\\
\mbox{}\\
\mbox{c1=C1()~{\#}~C1~instance}\\
\mbox{c2=C2()~{\#}~C2~instance}\\
\mbox{}\\
\mbox{{\#}{$<$}/example8.py{$>$}}
\end{flushleft}\end{ttfamily}
\end{quote}

In this example the class \texttt{C2} has already a magic docstring. This means that
\texttt{C2} has to be enhanced both from \texttt{Logged} and from \texttt{Decorated}.
This is done by automagically creating a \texttt{DecoratedLogged} class
decorator:
\begin{verbatim}>>> from example8 import C1,C2,c1,c2 
<class C2[DecoratedLogged]> created\end{verbatim}

The second line is printed because of the logging capabilities of \texttt{C2}.
Moreover, since \texttt{C2} is decorated too, the following will work:
\begin{verbatim}>>> assert C2.name() == 'C2'
>>> assert c2.name() == 'C2' \end{verbatim}

Idem for \texttt{C1}:
\begin{verbatim}>>> assert C1.identity(1) == 1
>>> assert c1.identity(1) == 1 \end{verbatim}


%___________________________________________________________________________

\hypertarget{the-implementation}{}
\section*{The implementation}
\pdfbookmark[0]{The implementation}{the-implementation}

This part can be safely skipped, unless you are a \emph{really} curious and
you want to know how the implementation works.

The module is rather short (less than 150 lines if you skip comments and
docstrings) but far from being trivial, since it is based on extensive 
usage of metaclass wizardry. Moreover,
it invokes the \texttt{safetype} module to solve metaclass conflicts, and
\texttt{safetype} contains 50 lines of \emph{really} deep metaclass wizardry.
\begin{quote}
\begin{figure}

\includegraphics{decorators.ps}
\end{figure}
\end{quote}

The main class-metaclass hierarchy is represented in the figure, where
boxes denote classes and ovals denote metaclasses; instantiation is
denoted by dashed green lines whereas inheritance is denoted by continuous
blue lines.

For instance, this is the Method Resolution Order for the \texttt{Decorated}
metaclass:
\begin{verbatim}>>> for i,c in enumerate(decorators.Decorated.__mro__): 
...     print i,c,"[%s]" % type(c).__name__
0 <class 'decorators.Decorated'> [MetaDecorator]
1 <class 'decorators.ClassDecorator'> [MetaDecorator]
2 <class 'safetype.safetype'> [type]
3 <type 'type'> [type]
4 <class 'decorators.Decorator'> [MetaDecorator]
5 <type 'object'> [type]\end{verbatim}

\texttt{Decorator} is the mother of all decorators. Its main purpose it to
provide the \texttt{MetaDecorator}
metaclass to all decorators.

The \texttt{safetype} metaclass, imported from the \texttt{safetype} module,
ensures that class decorators can generate classes without incurring 
in conflicts.

Since class decorators are metaclasses,
\texttt{MetaDecorator} is actually a meta-metaclass with respect to
instances of decorated classes. For this reason if
\begin{verbatim}>>> C=decorators.ClassDecorator('C',(),{})\end{verbatim}

then
\begin{verbatim}>>> C.__class__
<class 'decorators.ClassDecorator'>\end{verbatim}

but
\begin{verbatim}>>> C.__metaclass__ 
<class 'decorators.MetaDecorator'>\end{verbatim}

since the \texttt{C.{\_}{\_}metaclass{\_}{\_}} attribute is inherited from \texttt{Decorator}.

On the other hand, \texttt{MetaDecorator} is a simple metaclass with
respect to instances of decorated methods.

The implementation is relatively smart. Consider for instance the case of
the \texttt{logged} example. In that example class \texttt{D} was a subclass 
of a tricked \texttt{object} class, enhanced by \texttt{ClassDecorator}. Moreover \texttt{D} 
had a \texttt{Logged} docstring. Therefore it should have been an instance of a 
\texttt{ClassDecoratorLogged} metaclass. But logged was
a subclass of \texttt{ClassDecorator}, therefore it already had all the features
of \texttt{ClassDecorator} and it would have been redundant to create 
\texttt{ClassDecoratorLogged}, when``Logged`` would have been enough. 
So \texttt{Logged} was used and \texttt{ClassDecoratorLogged} was never created. 
All this magic is in the \texttt{safetype.remove{\_}redundant(*classes)} function.

The current implementation does not make any attempt of optimization and
there may be alternative implementations faster or more memory efficient.
At this experimental level I didn't care to explore about performances 
issues. Probably, they do not  matter unless one has to decorate
thousands or millions of functions and classes.

Finally, a word about bugs. The \texttt{decorators} module is fairly sophisticated,
therefore whereas I can guarantee that it passes my test suite (which involves
{\~{ }}200 tests, most of them extracted from the documentation you are reading), 
I cannot guarantee that it is correct. If somebody finds a bug or an 
unexpected behavior, please let me know and I will fix it or document it.
% References: 


%___________________________________________________________________________

\hypertarget{module-decorators}{}
\subsection*{Module \texttt{decorators}}
\pdfbookmark[1]{Module decorators}{module-decorators}

A module to implement pep318 (decorator syntax) via magic doctrings.
For the full documentation see
\href{http://www.phyast.pitt.edu/~micheles/python/decorators.html}{http://www.phyast.pitt.edu/{\~{ }}micheles/python/decorators.html} .


%___________________________________________________________________________

\hypertarget{id3}{}
\subsubsection*{Metaclasses}
\pdfbookmark[2]{Metaclasses}{id3}

\texttt{metaclass ClassDecorator(type,Decorator):}
\begin{quote}

Metaclass callable with one or three arguments, having its calling
syntax redefined by \texttt{MetaDecorator}.
\end{quote}

\texttt{metaclass Decorated(ClassDecorator):}
\begin{quote}

Metaclass which decorates the methods of its instances according
to the docstrings. It redefines \texttt{{\_}{\_}str{\_}{\_}} on
its instances and the default \texttt{{\_}{\_}str{\_}{\_}} on the instances of its
instances.

\texttt{{\_}{\_}str{\_}{\_}(cls):}
\begin{quote}

Redefine the printing representation of classes
\end{quote}
\end{quote}

\texttt{metaclass safetype(type):}
\begin{quote}

Overrides the \texttt{{\_}{\_}new{\_}{\_}} method of the \texttt{type} metaclass, making the
generation of classes conflict-proof.
\end{quote}

\texttt{metaclass MetaDecorator(type):}
\begin{quote}

Metaclass inducing a certain amount of magic on decorators:
\newcounter{listcnt1}
\begin{list}{\arabic{listcnt1}.}
{
\usecounter{listcnt1}
\setlength{\rightmargin}{\leftmargin}
}
\item 
Each time a decorator is defined in any module, it is stored in
MetaDecorator.dic and MetaDecorator.ls.

\item 
If the (method) decorator has a 'get' method, a '{\_}{\_}get{\_}{\_}' method
is automagically created as an alias to 'get'.

\item 
Decorators calls are dispatched to the decorator {\_}call{\_}
classmethod.

\end{list}

\texttt{{\_}{\_}call{\_}{\_}(dec,*args):}
\begin{quote}

This is the heart of the module. Infers the correct decorator class
from \texttt{dec} and the docstring and creates the correct decorator object.
Returns the original object if \texttt{dec} is the trivial \texttt{Decorator}
class and no docstring is found.
\end{quote}

\texttt{{\_}{\_}init{\_}{\_}(dec,*args):}
\begin{quote}

Update the metaclass attributes \texttt{dic} and \texttt{ls};
alias \texttt{get} to \texttt{{\_}{\_}get{\_}{\_}}.
\end{quote}
\end{quote}


%___________________________________________________________________________

\hypertarget{classes}{}
\subsubsection*{Classes}
\pdfbookmark[2]{Classes}{classes}

\texttt{class UnknownDecoratorError(Exception):}
\begin{quote}

The name says it all.
\end{quote}

\texttt{class Decorator(object):}
\begin{quote}

Instance of MetaDecorator and mothers of all decorators. When called
in the form \texttt{Decorator(obj)} with obj having a magic docstring, it returns
a decorated object; otherwise it returns the original object.
\end{quote}

\texttt{class staticmethod(MethodDecorator):}
\begin{quote}

Decorator, converts a function in a staticmethod
\end{quote}

\texttt{class MethodDecorator(Decorator):}
\begin{quote}

Descriptor class callable with a function or a descriptor object
as argument. It defines a default printing representation on method
decorators objects and a default \texttt{get}  method. All the rest is
provided by the metaclass \texttt{MetaDecorator}.

\texttt{get(self,obj,cls=None):}
\begin{quote}

aliased to {\_}{\_}get{\_}{\_} by the metaclass, to be overridden
\end{quote}

\texttt{{\_}{\_}str{\_}{\_}(self):}
\begin{quote}

Printing representation of kind {$<$}decoratorclass:functionname{$>$}
\end{quote}

\texttt{{\_}{\_}init{\_}{\_}(self,objfunc):}
\begin{quote}

objfunc is a decorator object or a function
\end{quote}
\end{quote}

\texttt{class classmethod(MethodDecorator):}
\begin{quote}

Decorator, converts a function in a classmethod
\end{quote}


%___________________________________________________________________________

\hypertarget{functions}{}
\subsubsection*{Functions}
\pdfbookmark[2]{Functions}{functions}

\texttt{enhance{\_}classes(magicstring='[ClassDecorator]'):}
\begin{quote}

Enhances all the classes in the caller module with a metaclass
built from the given magicstring.
\end{quote}

\texttt{remove{\_}redundant(bases):}
\begin{quote}

Returns a tuple of non-redundant base classes.
Given a sequence of base classes, a class is redundant if it is duplicated
or if it is implied by the others, i.e. it is an ancestor of at least one
of the other classes. For instance, if \texttt{C} is derived from \texttt{B}, in the
sequence \texttt{C,B} the class \texttt{B} is redundant, since all its features are
already provided by \texttt{C}. Therefore \texttt{B}
is removed and \texttt{remove{\_}redundant} returns the tuple \texttt{(C,)}:
\begin{verbatim}>>> class B(object): pass
...
>>> class C(B): pass
...
>>> import safetype; safetype.remove_redundant([C,B])
(<class 'C'>,)\end{verbatim}
\end{quote}

\texttt{compose(dclasses,*defaultclasses):}
\begin{quote}

Retrieves or creates a decorator for a tuple of decorators. If
defaults decorators are given, they get the precedence. It removes
redundant classes.
\end{quote}

\texttt{instance{\_}str(self):}
\begin{quote}

Redefines the printing representation of simple objects
\end{quote}

\texttt{getdec(magicstring="[Decorator]"):}
\begin{quote}

Given a magicstring describing a valid decorator name, returns the list
of its subclasses. By default returns the full list of decorators.
\end{quote}

\texttt{decorate(objdict):}
\begin{quote}

Takes an object with a dictionary and decorates all its functions
and classes according to their docstrings.
\end{quote}

\texttt{decorators{\_}in(docstring):}
\begin{quote}

Takes a docstring and returns a (possibly empty) tuple of decorator
classes.
\end{quote}

\end{document}

